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ABSTRACT 

Experiences with assessment at the following six 
diverse institutions are described: Clayton State College (Georgia), 
James Madison University, Kean College of New Jersey, King's College 
(Pennsylvania), Rhode Island College', and State University of New 
York (SUNY) College at Plattsburgh. Rhode Island College generates a 
personal learning plan for each new student using entry-level 
assessment of student abilities, attitudes, aspirations, financial 
aid needs, and high school achievements. SUNY, Plattsburgh, designed 
an Academic Development Survey to examine student study patterns, 
faculty-student interaction, and value and attitude development. 
J2imes Madison University attempts to link outcomes to environmental 
factors (e.g., the relationship between student leadership activities 
and academic achievement). King's College has a number of assessment 
activities, including the American College Testing program, capstone 
assessments in the major, and alumni surveys. Clayton State College 
ident^iied eight learning outcomes and established criteria for 
assessing them. At Kean College the initial value added orientation 
evolved to a concept of outcomes assessment. (SW) 
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It was March in Chicago, and there were fifteen of us 
around a table in the Hilton Towers, eating chicken salad 
packed in pineapple halves and feeling our way toward two 
hours of rich conversation; the subject was assessment. 

Tt was March, and for several months those of us with 
the Assessment Forum, a three-year FIPSE-funded project of 
the American Association for Higher Education, had been 
looking toward our June conference. The plan was to direct 
the conference toward teams prepared to move ahead with 
assessment on their own campuses, and we were naturally 
looking to sharpen our own sense of implementation. Having 
gathered a good deal of information over the previous two 
years about who was doing what with assessment, an<i where, 
we now needed to know more about the how-to. For me, 
personally , that meant opening a broader window on the 
field. Having recently come to the Forum from nine years at 
Alverno College, I was eager to see if my experience there 
was replicated elsewhere — what general principles and 
strategies of implementation (if any) could be identified. 
More importantly, what could the Foram do to make such 
information helpful to institutions now preparing to 
implement assessment? 

With this in mind, the Forum set out to identify 
iniititutions that might generate principles for implementing 
successful assessment. A natural first thought was to look 
toward those programs that jire most fully developed and have 
in some ways the most to teach. A second, more intriguing 
thought turned to campuses currently in the throes of 
implementation, those for whom the risks and headaches and 
happy insights are daily business. We were looking, that 
is, for a kind of "second wave" of assessment i/ork, not 
the WFxl-known pioneers (one thinks of Alverno, Northeast 
Missouri, the University of Tennesee at Knoxville- 
Miami-Dade), and not the next big wave just now beginning to 
explore assessr.ent, but a group somewhere between the two, 
experienced but still experimenting. 

Having encountered a number of schools thac fit that 
description, the Forum set about assembling them and 
eventually brought together in Chicago a manageable yet 
diverse group of six: Clayton State College, James Madison 
University, Kean College of New Jersey, King's College, 
Rhode Island College, and SUNY Plattsburgh. 

They talked; I listened. The stories I heard were 
frank and lively: stories of risks-taken and mistakes-made, 
breakthroughs and insights. No one at the table claimed to 
have the answers or the perfect model; yet all of them were 
in the midst of promising work, not just talking but do;ng 
assessment. What this paper attempts is to retell their 
stories, drawing out common principles, motifs — lessons, if 
you will--in hopes of moving other campuses toward 
successful assessment practices. 



I 

"What You're Doing Is Just Common Sense" 



"What you're doing is just common sense" is what Donald 
Farmer and George Hammerbacher of King's college often hear 
from visitors who come to learn about their ten years of 
work: the identification of eight transferable skills of 
liberal learning, the building of an entirely new core 
curriculum around those skills, and, more recently, the 
introduction of a comprehensive institutional assessment 
program. Common sense. It is, of course. 

Looking across the assessment horizon, one senses that, 
for many, assessment descends like some extraterrestial — 
suspicious, threatening, unknown, and most of all out of 
nowhere . Listening to this group, however, I was struck 
over and over by the good and common sense of it all. As 
Tom Koran of Plattsburgh put it, assessment is not something 
you start with, it*s a strategy for accomplishing other 
things, for improving the quantity and quality of learning. 
It is, you might say, a response to some rather obvious, 
common-sensical questions an institution might ask of 
itself: 

Who are our students and why do they come here? 

What do they know when they arrive? 

What should a graduate bp like? 

How do students change ana why? 

What factors lead students to withdraw? 

What do employers say about our graduates? 

How do students talk about their own learning? 

Is there a better way to organize the curriculum? 

What are the strengths of the institution? 

How could we do better? 

These are not, of course, the only questions possible; there 
are important issues of research and scholarship and public 
service to be addressed when looking at the quality of an 
institution. The point is simply that the questions which 
drive one to assessment are common ones, which come from 
insid e the institution, not from outer space, and have, in 
fact, a remarkable element of the ordinary and mundane about 
them. 

The "common sense" of. King's is typical in this regard. 
For Rhode Island College it was wanting to know more about 
how students changed that led several years ago to the 
formation of a seven-school consortium to measure "value 
added." Once into the project^ the College became 
increasingly aware that assessment, for them, had less to do 
with measuring gain for its own sake than with using results 
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to address some basic institutional concerns. With its 8000 
non-residential students, many of whom work while attending 
school, Rhode Island wanted to improve the quality of 
student advising, to go beyond tha usual guidance with 
course selection to address concerns like time management 
and balancing school with outside commitments. Assessment, 
Pat Soellner explained to the group, provides the vehicle 
for addressing that need. Using entry-level assessment of 
student abilities, attitudes, aspirations, financial aid 
considerations, values, and hig^ school achievements, the 
college generates (with the help of a computer) a "personal 
learning plan" for each new student. Eventually students 
will be able to update and adapt their plans each semester. 
Common sense. 

Assessment at SUNY Plattsburgh and James Madison can 
also be seen in this light. Like Rhode Island, Plattsburgh 
has, from the beginning, been interested in value added. 
Although wanting in a general way to learn more about what 
happens to their students, they soon found themselves drawn 
by data to what Tom Moran calls "soft spots," areas which 
seem to call for further attention--f or instance, the 
quality of student Interaction outside the classroom. To 
probe this and related areas of student experience, 
Plattsburgh has designed an "Academic Development Survey" 
that looks at student study patterns, faculty-student 
interaction, value and attitude development ("my experiences 
at Plattsburgh have made me more sensitive to issues of 
racial eqjality"), and more. Wanting to know more about a 
broad range of student experiences also drives assessment at 
James Madison. Like most programs, James Madison looks at 
knowledge and ability outcomes but, Dary Erwin explains, 
"It*s not enough to assess where students are ; we need to 
find out why . " Thus Erwin 's Student Assessment Office 
attempts to link outcomes to environmental factors, asking, 
for instance, about the relationship between student 
leadership activities and academic achievement, and the 
impact of financial aid on cognitive development. Again, 
the point is that assessment comes from (and no doubt 
raises) questions v/ithin the institution; it's not an add-on 
but an integral piece in a larger context. 

That this common sense view of assessment so clearly 
prevails in this group of six schools may itself be a 
function of context — the fact that these six schools don't 
see state level mandates driving their work. This is not to 
say they're unaware of what's happening in Florida and 
Missouri and Colorado and South Dakota — states where 
assessment is mandate-driven. Indeed, several of then- 
benefit from action in their own states. Kean has a state 
challenge grant to do what they're doing. James Madison is 
a pilot institution in Virginia's plan to upgrade 
undergraduate education. In every case, however, the 
assessment stories told in Chicago describe 
internally-generated responses to institutional concerns. 
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No doubt those with a public policy b^nt would question 
this view, suggesting thaf "second wave" assessment programs 
don't just suddenly and iiidependently arise but, rather, 
reflect t^le power of external calls for accountability. 
Indeed, Rhode Island describes itself as paving the way, or 
setting a model, for the eventual possibility of state 
action. But the fact is that these six schools have 
accomplished a good deal, and they perceive themselves as 
driven primarily by internal needs, not external forces. 
It's tempting to draw a conclusion here. 

At the very least, one might conclude that promising 
assessment ventures understand assessment as a means to an 
end, not an end in itself. Coming, as a it does for many 
institutions, as a call for public accountability, 
assessment is easily seen as a project in measurement. It 
was not, however, proving but improving that focused the 
stories told in Chicago. For each of the six schools, 
assessment is a way of exploring and making decisions about 
internally-generated questions and concerns: assessment as 
a possible road to better advising; a way of knowing more 
about--and therefore being better able to upgrade--the 
quality of student life outside the classroom. Others have 
seen it as a vehicle for examining patterns of withdrawal or 
transfer. Some (including elite, private institutions) turn 
to assessment as a way of better articulating what they do 
for students, thus improving recruitment and justifying 
costs . 

Perhaps the key word here is context, a term that 
surfaced over and over in the Chicago conversation. 
Curriculum, teaching strategies, admissions, advising, what 
happens in the residence halls, financial aid--assessment 
both follows from and calls for attention to each of these 
areas. It is not, that is, something one picks off a shelf 
or sticks in an office down the hall but part of an on-going 
account of the nature and quality of the institution. 

If there was one message chat stood out in the Chicago 
meeting, it was this contextual vision, this call to see 
assessment in the larger institutional context from which 
it takes — and to which it gives — meaning and direction. 

As one of the Chicago group put it: "What one takes on 
on with assessment is not a discrete 'assessment project, ' 
but a larger vision of what the institution and its students 
can be at their best." 



II 

"It Looks a Whole Lot Better on Paper, Doesn't It?'* 



Like everyone at the table in Chicago, Don Lumsden and 
Mike Knight of Kean College devoted some of their early work 
to exploring successful assessment ventures elsewhere. When 
they went to the University of Tennessee at Knoxville, they 
got not only good advice but (from Homer Fisher, Vice 
Chancellor for Business, Planning, and Finance) a line they 
occasionally like to quote: "It looks a whole lot better on 
paper, doesn't it?" Understood as part of a larger 
institutional context, assessment may be common sense, but, 
as Knight and Lumsden point out, common sense is not so 
common. Moving assessment from paper to practice is no 
cinch either. 

First, as everyone at the table testified, assessment 
time frames have a way of getting out of hand. One need 
only loc,< at the Tennessee program or at Northeast Missouri 
or Alverno to note that assessment swallows semesters. 
King's has in place an impressive array of assessment 
activities — the ACT COMP at entrance and graduation, 

capstone assessments in the major, alumni surveyj to name 

just a few — but King's has been at it for a decade. Clayton 
started six years a^o, first identifying eight institutional 
learning outcomes, then establishing criteria for assessing 
them. Like King's, they invested heavily in faculty 
development, generating a shared understanding of and 
responsibility for student learning in areas like critical 
thinking and valuing. Only then did they turn to curricular 
questions; and only recently, with the foundation (outcomes 
and criteria) securely in place, have faculty been ready to 
move ahead with actual assessment. 

State level mandates which call for assessment "up and 
running" in six months deny this most basic fact. We're 
talking about a decade of work^ not a little project in the 
dean's office but a way of thinking about quality that 
transforms the whole institution. 

And of course "the whole institution" includes faculty. 
All the most successful programs, as well as common sense, 
say this is essential. For assessment that will make a 
difference for student learning (and there, after all, is 
the acid test) there's no point in proceeding without 
faculty support and involvement; it's faculty who will be 
largely responsible for whether data lead to improvements. 
Unfortunately, even the most enthusiastic faculty may not be 
able to jump quickly and effectively into assessment. Most 
are not trained to think about outcomes beyond their own 
programs. Turf becomes an inevitable issue. Faculty task 
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forces on assessment take time (more than often 
anticipated), and that means time away from other agendas. 
Lectures need giving. Professor X needs to finish her book 
by tenure tinie. 

In this sense, the notion of institutional context is 
not only a source of meaning and direction in assessment 
but, ironically, a potential obstacle. The extent of 
changes can look overwhelming ev^n to the most dedicated 
assessment proponent. First the institution needs clearly 
articulated, agreed upon outcomes (as Ernest Boyer has said, 
one has no business even thinking of assessment until goals 
are clear); clarifying outcomes almost certainly entails 
rethinking in the disciplines (at Alverno, outcomes 
connected to the larger curriculum forced me to think of my 
discipline, English, in ways beyond the easy canon-bound 
definition); and then, of course, new outcomes call for 
new, more active and involving teaching strategies. 
Faculty development will almost certainly be called for. 
Committees proliferate. Semesters pass. 

As one member of the Chicago group said: once you're 
into assessment, you're into it forever. It's an on-going, 
cyclical process: gathering information, scrting it out, 
interpreting results, making changes — until you're back at 
the beginning, gathering information to assess the impact of: 
changes made. Looking down a road like that, who (faculty 
or otherwise) wouldn't hesitate? 

There's a chapter in the assessment story these people 
tell: a lot of characters are milling around; they turn to 
each other with plaintive, sometimes angry voices: "What's 
the matter here?" "Why do we have to do this?" "Why me?" 
It's a long long chapter... 



Ill 

The Paranoia of Exclusion and the Paranoia of Inclusion 



...a long chapter, but not the last one. The good news 
is the eventual transition from what Mike Knight and Don 
Lumsden of Kean call the paranoia of inclusion (Why me?) to 
the paranoia of exclusion (Hey, what about me? how do I get 
in on this?) , 

Kean's story is particulary to the point here. Not 
surprisingly, initial talk of assessment at Kean brought 
quick response from the faculty union (three of the six 
schools in Chicago are unionized), which issued a statement 
of concerns; they were, Don Lumsden says, "the right 
concerns." Indeed, it is in part out of those concerns 



found, may well be "the right concerns." Indeed, 
faculty questions make fertile soil for an assessment 
program. What is it faculty want to know about their 
students? their classes? What do they perceive as 
institutional "soft spots"? Even a quickly 
brain-stormed list of faculty questions might go a ng 
way toward shaping a campus assessment agenda. 



DO be careful about connecting assessment to some 
aspects of the context. Almost certainly faculty will 
be concerne*i about the possibiity of assessment 
connecte'^ to evaluation and promotion. If assessment is 
a story, this is a chapter called "kiss of death." 



DO enlist student support. Pat Soellner reports that 
Rhode Island observed greater faculty interest when 
student support for assessment was clear. Having at 
first sought such support informally, the College is 
currently hiring fifteen student assessment consultants, 
who will help shape the "learning plans" mentioned 
earlier, share their experience of the ACT COMP, and 
generally provide a student's eye view of institutional 
quality. At James Madison, too, efforts have been 
made to involve students and seek their support for 
assessment. Students hear about assessment at 
orientation; they read about it in the school newspaper. 



DO consider faculty development a central and on-going 
motif. It may be in the form of one-on-one consulting 
with staff from a central assessment office (an approach 
at Kean and James Madison); it may entail institutional 
workshops on, for example, defining outcomes; experts 
from elsewhere may be imported for the day. The bottom 
line, according to each of these six institutions, is 
on-going training and development across the 
institution, working toward a common sense of learning 
outcomes and a shared responsibility for achieving them. 
King's College spent five years on faculty development 
in such areas as writing across the curriculum; next 
came a fundamental remaking of the core curriculum; only 
then did Academic Dean Donald Farmer turn to talking 
about assessment itself. 



IV 

A Rose By Any Other Name? 




ShakesDeare may be right about roses--they smell as 
sweet by any name--but in higher education, labels matter. 
I recently heard Zelda ^amson report on a project where she 
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worked with faculty to improve the quality of teaching. 
Several otherwise dedicated teachers couldn't bring 
themselves to answer a questionnaire on teaching critical 
thinking skills because its language offended them; "I just 
want tc teach students to think, period!" was one woman's 
response. Another reported being unable to read cerjain 
higher education reports on teaching because "they're 
boring . " 

A similar semantic thorniness presents itself in 
working with assessment. For several in the Chicago group, 
institutional resistance to assessment, particularly from 
faculty, was first and foremost a resistence to language. 

In part the problem is a gap between faculty and 
administrative discourse . Maybe that ' s a give^* . In 
addition, discourse varies from discipline to discipline, 
with fundamental implications for talking about assessment. 
The psychology department, for example, may be perfectly 
happy with the word "assessment"; it's familiar, a part of 
the context in which they work. On the other hand, such 
familiarity may bring with it a narrower, more technical 
definition than makes sense in an institution-wide context. 
The language of outcomes and performance criteria and 
educational objectives (all a part of the assessment 
lexicon) may delight the education department but scare the 
pants off the historians. For King's College the most 
vigorous resistance came from the humanities. "I told them 
to give me the language," says Farmer, "but they didn't have 
it either." Now George Hammerbacher , Chair of the English 
Department, describes plans for a "rising junior exam, 
fondly referred to as the ' sophomore- junior project.'" 

With its obvious i nput -output , producti on-model 
derivation, "value added" poses particular semantic problems 
as one moves from the economics department to a wider range 
of disciplines. Indeed, at Kean, support for assessment 
increased markedly when the intitial value added language 
and or i en tat i on evolved to a concept of outcomes 
assessment--the language of "learn^.ng goals . " The point is 
not that there's a right language and a wrong one (Jerry 
Supple at Plattsburgh considered droping value added only to 
find the faculty task force inclined to keep it) but that 
reactions to assessment may, particularly at first, be 
reactions primarily to language. 

Language questions may also arise at a later stage of 
assessment work, when it comes to feeding data back to 
faculty. Indeed, that very phrase is enough to stop some 
department ears. Jim Doig, himself a leader in Clayton's 
assessment work, bristles more than a little bit: "I'm a 
philosopher. Data don't interest me, I don't like data, 
don't ask me to deal with data." He smiles as he says this, 
but there may be a :->erious point to be made. I flashed 
back, maybe five years, to faculty development jessions at 
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Alverno--being shown beautiful three-color overheads of data 
on student development, followed by soggy sileice from 
faculty who were asked to respond. Later Marc.a Mentkowski 
and her staff in the Research and Evaluation Office hit on 
the notion of audio-taped student interview excerpts. For 
many faculty, the language of individual student experience 
spoke more clearly than the language of aggregate data, and 
silence was replaced by vigorous discussion of implications 
of assessment results for teaching and curriculum 

The point of course is not that audio-taped interviews 
are superior to charts or that "sophomore-junior project" is 
better than "value added" but that language is a snag to 
watch for. And the lesson here (though "lesson" may not be 
the best word) has something to do with not staking too much 
on particular labels and keeping an ear out for useful 
alternatives. Language, like other dimensions of 
assessment, should fit the context. 



V 

Ice Cubes and Gardens 



"So one day we looked at each otner and said: 'Hey, 
what we've got here is an institutional change project.'" 
That's how Don Lumsden and Mike Knight describe a pivotal 
moment in their work at Kean. In fact, institutional 
change, how to understand and manage it, was a motif running 
throughout the Chicago conversation. It took shape in two 
distinct but equally suggestive metaphors: the ice cube and 
the garden. 

For Lumsden and Knight, it was the former. Having done 
their homework on assessment and called in the usual round 
of experts, they began to see a bigger picture and enrolled 
in a workshop on managing change. What they learned there 
was the ice cube model. 
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According to this model, institutional change moves 
through three stages: unfreeze ^ the slow unlocking of 
tightly held positions and attitudes; change ^ the 
restructuring and/or incorporation of new elements; and 
refreeze , the firming uo ( or institutionalization) of 
chanaos made. The ice cube is not, I suspect, the most 
soph^' icated model around, but for Kean college it provided 
an iU|jortant contrast to what Lumsden and Knight described 
as the "ice sculpture" approach, that is, carving away at a 
hard structure, hoping to change it from the outside. As an 
element firmly embedded in a larger context, assessment may 
require more radical strategies for change. 

Experience at King's College supports this view. One of 
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tne topics I had asked the Chicago group to think about in 
advance was "the riskiest thing you did." At King's, it was 
Don Farmer's fiat that "no course presently taught will be 
part of the new curriculum" (an attempt to forestall 
departmental turf wars, and a success). It was, if you 
will^ an "unfreeze" maneuver, a working from the inside out 
rather than chipping away, ice-sculpture style. 

The rerreeze portion of the ice cube model seems more 
problematic. At any rate a second metaphor for change also 
emerged. Whereas freezing suggests a firm, solid structure, 
some in this second wave group spoke in more organic terms. 
There were warnings against too-rigid plans and objectives, 
particularly in the early stages. For several, it made 
sense to begin with some general sense of direction but then 
to cultivate multiple options and scenarios, asking 
departments to generate "wish lists," developing as many 
experiments as possible ... .The metaphor h're is not freezing 
but growth, a garden model. 

In its clearinghouse and consulting function, the AAHE 
Assessment Forum enc ^nters issues of institutional change 
over and over. Regardless of the model, one theme emerges, 
as it did in Chicago: start down one road and almost 
inevitably assessment will take you places you didn't plan 
to go. Entry level assessment is a prime example. Knowing 
more about entering students, one begins to fiddle with the 
freshman curriculum, looking for a better match. When that 
happens, assessment down the line suddenly rears its head; 
one wants to see results of the changes. And so it goes. 
Plattsburgh is a case in point. Having begun with the 
Northeast Missouri State model, they quickly began to move 
away from it, discovering they were more interested in the 
side effects of measuring gain (a centerpiece of the 
Missouri program) than in measurement itself. 

In some sense, each of these programs started one place 
and ended up another. Research on effective management 
shows the value of a carefully thought out plan and also the 
stupidity of sticking to plans too rigidly when 
circumstances change. The same, the Chicago group suggests, 
is true of assessment. 



VI 

"Anything We Did Would Be Imperfect" 



Long-time student of assessment Peter Ewell talks about 
the fact that academics are by nature inclined toward 
inf ornvation; they love it, they spend their lives 
generating, interpreting, refining it. The corollary to 
this — and here's the rub^-is a deep abhorrance for acting 
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before all the facts are in. Assessment demands a different 
mentality, a recognition, as the Plattsburgh people put it, 
that "anything we did would be imperfect." 

And here we come to assessment in its more technical 
aspect of testing and measurement. Having passed through a 
stage of initial conceptualization (determining purposes for 
assessment, focus, uses of data...), one soon runs up 
against the problems of instrumentation. One may, after 
all, have outcomes in mind but no appropriate instruments at 
hand. Some, like the GRE, are too difficult for many 
assessment purposes; others, like the much-used ACT COMP (a 
choice at Rhode Island, Plattsburgh, and King's, though none 
uses it exclusively) yield reliable aggregate data but 
little in the way of individual student feedback. And what 
about faculty-designed instruments? They seem to offer a 
better match with specific program goals, and their design 
represents a further step in faculty involvement, but what 
does it take to validate such instruments? Who should do 
so? Is it worth the work? 

Both home-grown and store-bought instruments also 
raise validity questions when it comes to student 
motivation. To borrow a phrase, you can lead a student to 
assessment, but you can't make her do her best, a problem 
intensified by the longer, more accurately diagnostic 
instruments one would, in theory, do best to use. Some 
institutions are experimenting with incentives — everything 
from cash to movie tickets. No one in the Chicago group 
offered up a magic solution to the problem of student 
motivation; there was, however, strong consensus that 
institutions must find ways to talk to students about 
assessment, not to bribe but to invest them in a process 
they see as related to the quality of their own education. 
. An essential first step is making sure students receive 
feedback — promptly, clearly, and in terms related to their 
own experience. 

Beyond questions of validity are those of utility. To 
what extent will this or that instrument yield data that 
suggest specific prograru improvements, or changes in 
teaching strategies? (The answer seems to be: not many.) 
And how does one decide these things given the newness of 
the field? What about cost? Logistics of administration? 
Again, it often "looks better on paper." 

Judging from Chicago group, there are essentially two 
schools of thought on how to handle such technical 
questions, and both make good sense. James Madison decided 
to confront questions of technical validity head-on, 
early-on, by making expertise in these areas readily 
available. As director of the Student Assessment Center, 
one of Dary Erwin's jobs is to advise faculty on technical 
testing and measurement questions, to keep people from 
coming to wrong conclusions. "What, for instance," he asks. 
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"can we safely conclude from data derived from a 
self-selected sample of students (probably the more serious 
ones) who feel like showing up on Saturday morning?" One of 
the next big issue? in assessment nationally, Erwin 
predicts, wi 11 be technical measurement 
questions . 

Certainly no one in Chicago took issue with Erwin's 
emphasis on measurement. However, a second school of 
thought was also evident. For Kean College, validity 
questions are crucial, yes, but perhaps less urgent than at 
James Madison. Kean's approach has been to set temporarily 
aside the search for perfect information, to let faculty 
become accustomed to new ways of thinking about teaching and 
learning that follow from assessment, then to raise more 
technical measurement questions. Plattsburgh, too, sees 
validity as important, but, for now at least, the emphasis 
is less on data in itself than as a stimulus to faculty 
dialogue and reflection. 

In some ways the important point may be less what one 
does than the need to do something . Yes, everything will be 
imperfect, it's better on paper, easier said than done..., 
but a common feature of the Chicago group was that everyone 
had done something regardless. There was a kind of "jump in 
and see what happens" mentality. Rhode Island's Pat 
Soellner talked about their plan to generate individual 
student advising and study plans using, among other things, 
the ACT^COMP (usually not recommended for individual student 
feedback). And no, she admits, they're not sure it will 
work, but they are sure to learn something. The assessment 
program at King's has, in a sense, raised uncertainty to a 
principle of organization. Since no amount of research or 
consultation could ensure that one approach would be more 
valid or useful than another, departments have what Don 
Farmer calls "100 experiments in progress." 

In a sense, the theme here is means and ends. If your 
intended end is perfect data, no program of assessment will 
satisfy you. Recognizing that the product of assessment 
will have certain inevitable imperfections, however, one 
begins to attend to process--to new kinds and levels of 
faculty discourse, greater attention to student learning, 
more expl icit focus on teaching strategies, a stronger sense 
of institutional identity. In this way, assessment may 
have less to do with measurement than with on-going 
institutional self -evaluation and improvement. 
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VII 

Principles, Lessons, Ethos 



This paper began with a boast about the quality of the 
conversation on which it's based, and perhaps the richness 
resulted from no one thinking they knew all the answers. 
Quite the contrary. In discussing the purpose of their 
gathering, the group w&s cautious about offering up anything 
like "lessons." No cookbook. No 

six-easy-steps -to-perfect-assessment . Indeed, discuss ion 
suggested over and over that taking risks, making mistakes 
is a crucial part of the process, and that the process 
itself, the collective thinking through what to do and how, 
may be the most potent, transforming aspect of assessment. 

Nevertheless, from where I sat, a good number of 
transferable principles and lessons emerged from the Chicago 
conversation. They're best understood in the context of 
stories told, but readers may find it helpful by way of 
summary to see them drawn out and set forth in big, bold 
"two-by-four style . " 

- Assessment follows from internally-generated, 
common-sensical questions and concerns, things you want 
to know more about. This isn't to discount the force 
of external demands for accountability, but it is to 
suggest that assessment will work best where those 
demands are somehow translated into genuine internal 
institutional concerns. In short, assessment is a 
means to an end the institution cares about. A 
corollary to this is: Beware of measurement for 
measurement ' s sake . 

- Anticipate roadblocks, and see them as a natural and 
inevitable part of the terrain. Assessment entails 
fundamental changes in the institution; it's 
conceptually and logically complicated. Most 
steps — if they're to mean anything — will take longer 
than anticipated. Leave room in plans for adjustments 
and redirection. Remember that it may sometimes "look 
better on paper . " 

- Involve people. If improved teaching and learning is 
the goal, not only top-level administrators but 
faculty need to be " stake -holders " in assessment. 
Listen to their initial concerns, use their questions 
to build an assessment agenda, involve them in 
interpretation of data. Without faculty support, even 
the most suggestive data won't lead to improvement. 
Students, too, need to be invested in assessment. Help 
them understand the process and its purposes. 

- Watch your language. Resistence to assessment is often 
resistence to language more than to its underlying 



concept and purpose. Help departments find their own 
language for talking about assessment. Avoid jargon. 
Work for "common sense . " 

- Think of assessment as an institutional change project. 
Like every such project, ir has its own unique 
wrinkles, but keep in view broader principles of 
effective change. Perhaps most important: stay 
flexible, and listen to the institution. 

- Do something even though it won't be perfect. Keep your 
eye on questions of validity and reliability but 
recognize, too, that it*s in the process of working 
through admittedly imperfect plans for assessment that 
crucial changes may occur: outcomes are explicitly 
debated, questions about teaching get asKed, faculty 
find new ways to think about their purposes, 
departments talk to each other, institutional identity 
grows stronger .... So do something, and whatever it is 
talk about it. 



These are not, perhaps, earth-shattering principles. They 
are, however, strategies without which assessment is less 
likely to be implemented successfully. Most important, they 
follow from what I can only call an ethos that pervaded the 
Chicago conversation and was for me it*s most powerful 
message: that assessment is above all a way of thinking 
about quality that comes from self -consciousness about our 
purposes and a willingness co examine, question, and alter 
what we do and who we are as educators . 
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ADDITIONAL RESOURCES 

AAHE ASSESSMENT FORUM 



The following resources are available for purchase from the AAHE 
Assessment Forum: 

I Resource Packet Five Papers $15.00 

-"Assessment, Accountability, and Improvement: Managing the 
Contradiction," P. Ewell 

—"Assessment and Outcomes Measurement: A View from the 

States/' C. Boyer, P. Ewell, J. Finney, J. Mingle 

--'The External Examiner Approach to Assessment," B. Fong 

—"Six Stories: Implementing Successful Assessment," P. 

Hutchings 

-"Thinking About Assessment: Perspectives for Presidents and 
Chief Academic Officers," E. El-Khawas and J. Rossmann 

2. Thi^ Presentations: $aOO 

from the 2nd National Conference on Assessment in Higher Education 
—Lee S. Shulman — "Assessing Content and Process: 
Challenges for the New Assessments" 
-Virginia B. Smith - "In the Eye of the Beholder: 
Perspectives on Quality" 

-Donald M. Stewart - "The Ethics of Assessment" 

3. Audio Tape: Shulman Address $9.00 

"Assessing Content and Process: Challenges for the New 
Assessments," Lee S. Shulman 

4 Address Roster of Denver Conference Participants $3.00 

Available Soon: 

National Directory on Assessment in Higher Education 

To order items indicated abov(^ and for more information about future 
Assessment Forum resources, services, and activities, contact: Patricia 
Hutchings, Director, AAHE Assessment Forum, One Dupont Circle, Suite 
600, Washington, DC 20036; 202/293-6440. 

Orders under $25 must be prepaid. Allow four weeks for delivery. 
Postage and handling is included in prices quoted. 



MAKE CHECKS PAYABLE TO AAHE ASSESSMENT FORUM 



